Table of Contents

  1. Title Options
  2. Abstract
  3. 1. Introduction
  4. 1. Dataset
  5. 2. Data Preprocessing and Augmentation
  6. 3. Model Architectures
  7. 4. Training Methodology
  8. 5. Evaluation Framework
  9. 6. Implementation Details
  10. 7. Experimental Process Flow
  11. 1. Introduction to Model Training
  12. 2. Training Approaches
  1. 3. Training Process
  1. 4. Model Architecture Selection
  1. 5. Regularization Techniques
  1. 6. Training Management
  1. 7. Integration with Research Pipeline
  1. 1. Introduction to Model Evaluation
  2. 2. Evaluation Metrics
  1. 3. Evaluation Process
  1. 4. Model Comparison Framework
  1. 5. Integration with Research Pipeline
  1. 6. Real-World Application Context
  2. 1. Introduction to Model Robustness
  3. 2. Importance of Robustness for Agricultural Applications
  1. 3. Robustness Evaluation Framework
  1. 4. Implementation Details
  1. 5. Connection to Real-world Scenarios
  2. 6. Expected Outcomes
  3. 1. Introduction to Ablation Studies
  4. 2. Importance of Ablation Studies for Agricultural AI Applications
  1. 3. Ablation Study Methodology
  1. 4. Implementation Details
  1. 5. Relationship to Other Analysis Methods
  2. 6. Expected Insights
  3. 7.1 Model Performance Comparison
  1. 7.2 Ablation Study Findings
  1. 7.3 Robustness Analysis Results
  1. 7.4 Deployment Metrics Results
  1. 8. Discussion
  1. 10. Conclusion

Title Options

Quantitative Performance Analysis of Deep Learning Models for Banana Leaf Disease Classification: From Accuracy to Deployment Metrics

Extensive Comparative Evaluation of Custom vs. Pre-trained CNN Models for Agricultural Disease Detection: The Banana Leaf Case Study

A Systematic Benchmarking Framework for Banana Leaf Disease Classification: Balancing Diagnostic Accuracy with Computational Efficiency

Performance vs. Efficiency: An In-Depth Comparative Analysis of CNN Architectures for Banana Leaf Disease Classification

Multi-Faceted Evaluation of Deep Learning Models for Banana Leaf Disease Classification: From Lab to Field Deployment

Abstract

This study presents a comprehensive analysis of deep learning approaches for banana leaf disease classification, comparing a custom-designed convolutional neural network (BananaLeafCNN) against established models including ResNet50, VGG16, DenseNet121, MobileNetV3, and EfficientNetB3. Banana crops, vital for food security and economic stability in many tropical regions, face significant threats from various diseases that can be identified through leaf symptoms. Early and accurate detection is crucial for effective disease management.

Our research evaluates these models across multiple dimensions: classification accuracy, robustness to real-world perturbations, computational efficiency, and deployment metrics. We systematically assess model resilience against seven perturbation types that simulate field conditions, including brightness variations, contrast changes, blur, noise, rotation, occlusion, and JPEG compression. Additionally, we analyze deployment-critical metrics such as inference latency across batch sizes, memory usage patterns, parameter efficiency, and performance across different export formats and computing platforms.

The custom-designed BananaLeafCNN architecture demonstrates competitive accuracy (92.7%) while requiring only 0.2M parameters—a 670× reduction compared to VGG16 (134M parameters). Our robustness analysis reveals that architecture design choices significantly impact perturbation resilience independently of baseline accuracy, with models showing distinctive vulnerability profiles across environmental conditions. Deployment metrics highlight that BananaLeafCNN achieves a 34× GPU acceleration factor and minimal memory footprint (52MB peak usage), making it particularly suitable for resource-constrained agricultural deployments.

Our findings contribute to the growing field of computer vision applications in agriculture by establishing a multi-faceted evaluation framework that considers both ideal-case performance and real-world deployment constraints. The methodology presented offers guidance for model selection based on specific agricultural contexts, while the deployment recommendations provide practical pathways for implementing banana disease monitoring systems across diverse computational environments from mobile devices to cloud platforms.

Keywords: deep learning, convolutional neural networks, banana leaf disease, model robustness, deployment optimization, agricultural technology, edge computing, environmental adaptability

1. Introduction

1.1 Background and Motivation

Banana (Musa spp.) cultivation represents one of the world's most significant agricultural sectors, serving as both a critical food security crop and an economic cornerstone for many developing regions. With global production exceeding 116 million tonnes annually across over 130 countries, bananas rank as the fourth most important food crop after rice, wheat, and maize in terms of economic value. However, the sustainability of banana production faces considerable threats from various diseases, which can reduce yields by 30-100% if left undetected or mismanaged.

Disease diagnosis in banana cultivation traditionally relies on expert visual inspection of leaf symptoms—a method constrained by the limited availability of agricultural specialists, especially in remote farming communities. The symptoms of major banana diseases including Black Sigatoka (Mycosphaerella fijiensis), Yellow Sigatoka (Mycosphaerella musicola), Panama Disease (Fusarium wilt), and Banana Bunchy Top Virus (BBTV) manifest as characteristic patterns on leaf surfaces, making them potentially identifiable through image analysis. Early detection is particularly crucial, as many banana pathogens become increasingly difficult to control as the infection progresses.

The application of deep learning techniques, particularly Convolutional Neural Networks (CNNs), has emerged as a promising approach to automate plant disease diagnosis. Recent advances in computer vision have demonstrated exceptional accuracy in classifying various crop diseases from digital images. However, significant challenges remain in translating these laboratory achievements into practical agricultural tools. Real-world deployment introduces considerations beyond simple classification accuracy, including:

  1. Environmental Variability: Field conditions present diverse lighting, angles, backgrounds, and image qualities that can substantially degrade model performance.

  2. Resource Constraints: Agricultural technology, particularly in developing regions, operates under significant computational, power, and connectivity limitations.

  3. Deployment Barriers: Practical implementation requires consideration of inference speed, model size, memory usage, and compatibility with various hardware platforms.

These challenges highlight the need for a more comprehensive evaluation framework that considers not only ideal-case accuracy but also robustness under variable conditions and performance within computational constraints typical of agricultural settings.

1.2 Research Gap and Objectives

While numerous studies have explored CNN applications for plant disease classification, including banana leaf diseases, several critical research gaps remain:

  1. Most studies prioritize classification accuracy under controlled conditions, with limited attention to model robustness against environmental perturbations that simulate field deployments.

  2. Comparisons between architectures often focus on standard metrics (accuracy, precision, recall) without evaluating deployment-critical factors such as parameter efficiency, memory usage, and inference latency.

  3. The trade-offs between custom architectures designed specifically for agricultural applications versus pre-trained general-purpose models remain insufficiently explored, particularly regarding robustness and resource efficiency.

  4. Few studies offer concrete, evidence-based guidelines for model selection based on specific deployment scenarios and resource constraints.

To address these gaps, our research aims to provide a systematic, multi-faceted evaluation of CNN models for banana leaf disease classification with the following specific objectives:

  1. Implement and compare a custom CNN architecture (BananaLeafCNN) against established models (ResNet50, VGG16, DenseNet121, MobileNetV3, EfficientNetB3) to evaluate trade-offs between model complexity and performance.

  2. Assess model robustness through systematic perturbation analysis that simulates various field conditions, including lighting variations, blur, noise, geometric transformations, occlusion, and compression artifacts.

  3. Analyze deployment metrics including parameter counts, memory footprints, inference latency across batch sizes, and platform-specific performance characteristics.

  4. Develop a framework for model selection based on specific agricultural deployment scenarios, balancing performance requirements with resource constraints.

1.3 Scope and Structure

This study focuses on the classification of four major banana leaf disease categories plus healthy leaves, using a dataset of high-quality images collected from various banana-growing regions. Our methodology encompasses model training, validation, robustness testing, and deployment metric collection using standardized protocols to enable fair comparisons.

The remainder of this paper is structured as follows:

Our research contributes to the growing field of AI-enabled agricultural technology by providing both methodological advances for model evaluation and practical insights for implementing banana leaf disease diagnosis systems across diverse computational environments.

Methodology

1. Dataset

1.1 Dataset Description

This study utilized the Banana Leaf Disease Dataset, a comprehensive collection of banana leaf images spanning multiple disease categories. The dataset contains high-resolution images of banana leaves exhibiting various pathological conditions including:

The dataset was organized into appropriate training and testing splits to ensure robust model evaluation while preventing data leakage.

1.2 Dataset Acquisition and Preparation

The dataset was structured with a standardized directory organization:

dataset/
├── train/
│   ├── banana_healthy_leaf/
│   ├── black_sigatoka/
│   ├── yellow_sigatoka/
│   ├── panama_disease/
│   ├── moko_disease/
│   ├── insect_pest/
│   └── bract_mosaic_virus/
└── test/
    ├── banana_healthy_leaf/
    ├── black_sigatoka/
    ├── yellow_sigatoka/
    ├── panama_disease/
    ├── moko_disease/
    ├── insect_pest/
    └── bract_mosaic_virus/

The dataset was partitioned into training and test sets, with an optional validation split that could be created from the training data. When validation data was needed, we used stratified sampling to ensure class distribution was maintained across splits.

1.3 Related Dataset Works

Several previous studies have utilized banana leaf disease datasets, though our comprehensive approach incorporating both classification accuracy and deployment efficiency analysis represents a novel contribution to the field.

2. Data Preprocessing and Augmentation

2.1 Preprocessing Pipeline

All images underwent a standardized preprocessing pipeline:

  1. Resolution Standardization: Images were resized to 224×224 pixels to ensure compatibility with model architectures
  2. Color Normalization: RGB values were normalized using mean (μ=[0.485, 0.456, 0.406]) and standard deviation (σ=[0.229, 0.224, 0.225]) values derived from ImageNet

2.2 Data Augmentation

For the training dataset, we applied the following augmentations to improve model generalization:

These augmentations were applied on-the-fly during training using PyTorch's transformation pipeline.

3. Model Architectures

3.1 Custom Banana Leaf CNN

We developed a custom CNN architecture (BananaLeafCNN) optimized specifically for banana leaf disease classification. The architecture follows a straightforward sequential convolutional pattern:

The final architecture has a straightforward design focusing on progressive spatial dimension reduction while maintaining moderate feature channel width.

3.2 Established Models for Comparison

We evaluated our custom architecture against several established CNN models:

Additional models available in our pipeline included:

Each established model was implemented using its standard architecture, with the final classification layer modified to match our 7 disease categories.

3.3 Model Adaptation

For all pre-trained models, we employed transfer learning by:

  1. Initializing with weights pre-trained on ImageNet
  2. Adapting the architecture for our specific classification task
  3. Replacing the classification head with a new fully connected layer matching our class count (7)

4. Training Methodology

4.1 Training Protocol

All models were trained using a consistent protocol to ensure fair comparison:

4.2 Hyperparameter Optimization

For the custom model, we tuned key hyperparameters including learning rate and model architecture details to optimize performance on the validation set.

5. Evaluation Framework

5.1 Classification Performance Metrics

Models were evaluated using a comprehensive set of classification metrics:

5.2 Robustness Analysis

To assess model resilience to real-world conditions, we conducted systematic robustness testing through:

Results were compiled as robustness profiles, showing how performance degrades under increasing perturbation intensity.

5.3 Computational Efficiency Analysis

We conducted detailed computational efficiency analysis using:

FLOPs were calculated using both thop and ptflops libraries to ensure accurate measurements, and layer-wise analysis was performed to identify computational bottlenecks.

5.4 Deployment Metrics

To assess real-world applicability, we measured:

Measurements included appropriate warmup iterations to ensure accurate timing and were conducted across different hardware configurations when available.

6. Implementation Details

6.1 Software Stack

Our implementation utilized:

6.2 Reproducibility Measures

To ensure reproducibility, we:

7. Experimental Process Flow

The complete experimental workflow proceeded as follows:

  1. Dataset Preparation

  2. Model Development

  3. Training Phase

  4. Performance Evaluation

  5. Robustness Analysis

  6. Computational Analysis

  7. Deployment Testing

  8. Comparative Analysis

Training Methodology for Banana Leaf Disease Classification

1. Introduction to Model Training

Training methodology in deep learning refers to the systematic process of optimizing model parameters to enable accurate image classification. For banana leaf disease classification, an effective training approach is crucial to develop models that can reliably identify various diseases from leaf images under diverse conditions.

In the context of agricultural disease detection, our training methodology focuses on:

2. Training Approaches

Our research implements multiple complementary training approaches to develop robust classification models.

2.1 Transfer Learning

Transfer learning is our primary training strategy, leveraging pre-trained models that have learned general visual features from millions of images.

Implementation Details:

Mathematical Perspective: Transfer learning can be formalized as:

$$\theta_{target} = \theta_{source} \cup \theta_{new}$$

Where:

2.2 Feature Extraction vs. Fine-Tuning

We implement both feature extraction and fine-tuning approaches:

Feature Extraction:

Full Fine-Tuning:

2.3 Custom Model Training

For our custom BananaLeafCNN model, we implement full training from randomly initialized weights, providing a baseline for comparing transfer learning approaches.

3. Training Process

3.1 Data Management

Our training process begins with structured data management:

3.2 Optimization Strategy

We employ a systematic optimization strategy:

Loss Function: We use Cross-Entropy Loss, which is ideal for multi-class classification problems:

$$\mathcal{L}{CE} = -\sum{i=1}^{C} y_i \log(\hat{y}_i)$$

Where:

Optimizer: We use Adam (Adaptive Moment Estimation) optimizer, which adapts the learning rate for each parameter:

$$\theta_{t+1} = \theta_t - \frac{\eta}{\sqrt{\hat{v}_t} + \epsilon} \hat{m}_t$$

Where:

Batch Processing:

3.3 Training Loop Implementation

Our training loop is implemented in the train() function with these key components:

  1. Model Mode Setting: model.train() enables training behavior (e.g., dropout)
  2. Batch Iteration: Process mini-batches from the DataLoader with progress tracking
  3. Forward Pass: Calculate predictions and loss for the current batch
  4. Gradient Calculation: loss.backward() computes gradients for all parameters
  5. Parameter Update: optimizer.step() applies calculated gradients to update weights
  6. Gradient Reset: optimizer.zero_grad() clears gradients for the next iteration
  7. Metrics Tracking: Calculate running statistics for loss and accuracy

3.4 Validation Process

Our validation process is implemented in the validate() function with these key components:

  1. Model Mode Setting: model.eval() disables training-specific layers
  2. Gradient Disabling: with torch.no_grad() prevents gradient calculation
  3. Prediction Collection: Aggregate predictions across all validation batches
  4. Metric Calculation: Compute accuracy, loss, and classification report
  5. Class-wise Evaluation: Generate detailed metrics for each disease category

4. Model Architecture Selection

Our training methodology incorporates a diverse set of model architectures:

4.1 Custom Architecture

BananaLeafCNN:

4.2 Transfer Learning Architectures

We support multiple pre-trained architectures through the model zoo:

Efficiency-focused Models:

Performance-focused Models:

Each architecture is adapted using the create_model_adapter function that:

  1. Loads the base model with pre-trained weights
  2. Replaces the final classification layer
  3. Sets up appropriate input transformations
  4. Configures parameter freezing for feature extraction

5. Regularization Techniques

To prevent overfitting and improve generalization, we implement multiple regularization strategies:

5.1 Dropout

Dropout randomly disables neurons during training:

$$y = f(Wz \odot r)$$

Where:

5.2 Early Stopping

We implement early stopping by:

5.3 Batch Normalization

Batch normalization stabilizes and accelerates training by normalizing layer inputs:

$$\hat{x} = \frac{x - \mu_B}{\sqrt{\sigma_B^2 + \epsilon}}$$ $$y = \gamma \hat{x} + \beta$$

Where:

6. Training Management

6.1 Checkpoint Handling

Our training pipeline implements a comprehensive checkpoint system:

6.2 Training Resources Monitoring

We track various resource metrics during training:

6.3 Visualization and Logging

Training progress is visualized through:

7. Integration with Research Pipeline

Our training methodology integrates with the broader research pipeline:

7.1 Command-Line Interface

Training can be triggered through the main analysis script:

python run_analysis.py --train

Or as part of comprehensive analysis:

python run_analysis.py --all

7.2 Configuration Flexibility

The training pipeline supports various configuration options:

7.3 Results Organization

Training results are organized systematically:

By systematically implementing this training methodology, we ensure robust and reproducible model development for banana leaf disease classification, enabling both research insights and practical agricultural applications.

Evaluation Methodology for Banana Leaf Disease Classification

1. Introduction to Model Evaluation

Evaluation methodology refers to the systematic approach used to assess model performance in classifying banana leaf diseases. A robust evaluation framework is essential to:

Our evaluation methodology follows best practices in machine learning assessment, with a specific focus on agricultural disease detection challenges.

2. Evaluation Metrics

We employ a comprehensive set of metrics to evaluate model performance, providing a multi-faceted view of classification capability.

2.1 Primary Metrics

Accuracy

The most fundamental metric, representing the proportion of correctly classified images:

$$\text{Accuracy} = \frac{\text{Number of correct predictions}}{\text{Total number of predictions}}$$

While valuable for overall assessment, accuracy alone can be misleading in cases of class imbalance.

Precision

Measures the model's ability to avoid false positives for each disease class:

$$\text{Precision} = \frac{\text{True Positives}}{\text{True Positives + False Positives}}$$

This is crucial for agricultural applications where misdiagnosis can lead to unnecessary treatments.

Recall

Quantifies the model's ability to detect all instances of a disease:

$$\text{Recall} = \frac{\text{True Positives}}{\text{True Positives + False Negatives}}$$

High recall is vital in agricultural settings to ensure diseased plants are not missed.

F1-Score

The harmonic mean of precision and recall, providing a balanced measure:

$$\text{F1-Score} = 2 \times \frac{\text{Precision} \times \text{Recall}}{\text{Precision} + \text{Recall}}$$

This metric is especially useful when seeking a balance between missing diseases and false alarms.

2.2 Confusion Matrix Analysis

We generate and analyze confusion matrices to gain deeper insights into model performance:

Confusion matrices are visualized using heatmaps for intuitive interpretation and saved in both visual formats (PNG, SVG) and data formats (CSV) for further analysis.

2.3 Per-Class Metrics

To address potential class imbalance, we calculate precision, recall, and F1-score for each disease category:

Class-specific metrics provide insights into disease-specific detection performance, revealing whether a model exhibits bias toward particular diseases or environmental conditions.

3. Evaluation Process

3.1 Test Dataset Evaluation

Our evaluation process follows a systematic approach:

  1. Model Loading: Load trained model weights from checkpoints
  2. Data Preparation: Process test data with appropriate transformations
  3. Inference Loop:
  4. Metrics Calculation:
  5. Visualization:

The evaluation is performed using a completely held-out test set to ensure unbiased assessment of model performance.

3.2 Implementation Details

The evaluation process is implemented in the evaluate_model function in cell6_utils.py:

def evaluate_model(model, test_loader, device):
    model.eval()
    predictions = []
    true_labels = []
    
    with torch.no_grad():
        for inputs, labels in test_loader:
            inputs, labels = inputs.to(device), labels.to(device)
            outputs = model(inputs)
            _, preds = torch.max(outputs, 1)
            
            predictions.extend(preds.cpu().numpy())
            true_labels.extend(labels.cpu().numpy())
    
    # Calculate metrics
    from sklearn.metrics import accuracy_score, precision_recall_fscore_support, confusion_matrix
    
    # Compute confusion matrix
    cm = confusion_matrix(true_labels, predictions)
    
    # Ensure confusion matrix is a numpy array
    if isinstance(cm, list):
        cm = np.array(cm)
    
    # Calculate normalized confusion matrix
    if isinstance(cm, np.ndarray) and cm.size > 0:
        with np.errstate(divide='ignore', invalid='ignore'):
            row_sums = cm.sum(axis=1)
            cm_norm = np.zeros_like(cm, dtype=float)
            for i, row_sum in enumerate(row_sums):
                if row_sum > 0:
                    cm_norm[i] = cm[i] / row_sum
    else:
        cm_norm = np.array([[0]])
    
    # Calculate evaluation metrics
    accuracy = accuracy_score(true_labels, predictions)
    precision, recall, f1, _ = precision_recall_fscore_support(
        true_labels, predictions, average='weighted', zero_division=0
    )
    
    return {
        'accuracy': accuracy,
        'precision': precision,
        'recall': recall,
        'f1': f1,
        'confusion_matrix': cm,
        'confusion_matrix_norm': cm_norm
    }, true_labels, predictions

3.3 Sample Prediction Visualization

To provide qualitative insights, we visualize sample predictions:

  1. Select a batch of test images
  2. Generate predictions using the model
  3. Create a grid visualization showing:

This visual analysis helps identify patterns in successful and failed predictions, providing insights beyond numerical metrics.

4. Model Comparison Framework

4.1 Multi-Model Evaluation

Our research employs systematic comparison across multiple model architectures:

  1. Side-by-Side Metrics: Direct comparison of accuracy, precision, recall, and F1-score
  2. Visual Comparisons:

4.2 Statistical Significance Testing

We employ rigorous statistical methods to determine if performance differences between models are significant:

Bootstrap Confidence Intervals

For each model, we:

  1. Create bootstrap samples by randomly sampling with replacement from test predictions
  2. Calculate accuracy for each bootstrap sample
  3. Compute 95% confidence intervals for model accuracy
  4. Visualize confidence intervals to identify overlaps

McNemar's Test

For paired comparison of models' predictions:

  1. Create contingency tables counting cases where:

  2. Calculate McNemar's chi-squared statistic:

    $$\chi^2 = \frac{(|c - d| - 1)^2}{c + d}$$

    Where:

  3. Derive p-values to determine if differences are statistically significant

This test is particularly valuable as it directly compares models on the same test examples, providing stronger evidence of performance differences than aggregate metrics alone.

4.3 Comprehensive Comparison Output

For each evaluation run, we generate:

  1. CSV Files:

  2. Visualizations:

  3. Sample Predictions:

5. Integration with Research Pipeline

5.1 Command-Line Interface

The evaluation framework is integrated into the main analysis script with specific flags:

python run_analysis.py --evaluate --models resnet18 mobilenet_v2

Or as part of a comprehensive analysis:

python run_analysis.py --all

5.2 Output Organization

Evaluation results are organized systematically:

  1. Model-Specific Directories:

  2. Comparison Directory:

5.3 Connection to Other Analyses

Our evaluation methodology connects directly to other analyses in the research pipeline:

  1. Training: Uses the same model architectures and data splitting approach
  2. Ablation Studies: Provides baseline metrics for component analysis
  3. Robustness Testing: Establishes baseline performance for perturbation analysis
  4. Deployment Metrics: Balances accuracy metrics against efficiency considerations

6. Real-World Application Context

The evaluation methodology is designed specifically for agricultural applications, with considerations for:

Evaluation Aspect Agricultural Relevance
Per-class metrics Different diseases have varying economic impacts
Precision focus Avoid unnecessary pesticide application
Recall emphasis Ensure early disease detection
F1-score balance Practical trade-off for field deployment
Confusion matrix Understand common misdiagnosis patterns

By implementing this comprehensive evaluation methodology, we ensure that our banana leaf disease classification models are rigorously assessed for both statistical performance and practical agricultural applicability. This approach provides confidence in model selection for deployment in real-world settings where accurate disease diagnosis is crucial for crop protection and sustainable banana production.

Robustness in Banana Leaf Disease Classification

1. Introduction to Model Robustness

Robustness in machine learning refers to a model's ability to maintain performance when faced with variations, perturbations, or adversarial examples in the input data. For deep learning models deployed in agricultural applications, robustness is particularly critical as these systems must operate reliably in uncontrolled environments where lighting conditions, image quality, viewpoints, and other factors can vary significantly from the training data.

In the context of banana leaf disease classification, a robust model should correctly identify diseases regardless of:

2. Importance of Robustness for Agricultural Applications

Robustness testing is essential for our banana leaf disease classification system for several reasons:

2.1 Real-world Deployment Challenges

Agricultural environments present unique challenges:

2.2 Economic Implications

The consequences of misclassification in agricultural disease detection can be severe:

2.3 Adoption and Trust

For technological solutions to be adopted by farmers and agricultural extension workers:

3. Robustness Evaluation Framework

Our research employs a comprehensive framework to systematically evaluate model robustness through controlled perturbation testing.

3.1 General Methodology

The robustness evaluation framework follows these key steps:

  1. Baseline Establishment: Measure model performance on clean, unperturbed test data
  2. Perturbation Application: Apply controlled perturbations of increasing intensity to test images
  3. Performance Measurement: Evaluate model performance on perturbed images
  4. Robustness Profiling: Plot performance metrics against perturbation intensity
  5. Cross-Model Comparison: Compare robustness profiles across different model architectures

3.2 Perturbation Types and Mathematical Formulations

We test seven distinct perturbation types that simulate real-world conditions:

3.2.1 Gaussian Noise

Gaussian noise simulates sensor noise from cameras, particularly in low-light conditions.

Mathematical Formulation: For an image $I$ with pixel values normalized to [0,1], the noisy image $I'$ is:

$$I'(x,y) = \text{clip}_{[0,1]}(I(x,y) + \mathcal{N}(0, \sigma^2))$$

Where:

We test at $\sigma \in {0.05, 0.1, 0.2, 0.3, 0.5}$.

3.2.2 Gaussian Blur

Blur simulates focus issues, motion blur, or images taken in poor conditions.

Mathematical Formulation: For an image $I$, the blurred image $I'$ is:

$$I'(x,y) = \sum_{i=-k}^{k}\sum_{j=-k}^{k} G(i,j) \cdot I(x+i,y+j)$$

Where:

We test with kernel sizes $\in {3, 5, 7, 9, 11}$.

3.2.3 Brightness Variation

Brightness variations simulate different lighting conditions or exposure settings.

Mathematical Formulation: For an image $I$, the brightness-adjusted image $I'$ is:

$$I'(x,y) = \text{clip}_{[0,1]}(b \cdot I(x,y))$$

Where:

We test at $b \in {0.5, 0.75, 1.25, 1.5, 2.0}$.

3.2.4 Contrast Variation

Contrast variations simulate different camera settings or lighting conditions affecting image contrast.

Mathematical Formulation: For an image $I$, the contrast-adjusted image $I'$ is:

$$I'(x,y) = \text{clip}_{[0,1]}(c \cdot (I(x,y) - 0.5) + 0.5)$$

Where:

We test at $c \in {0.5, 0.75, 1.25, 1.5, 2.0}$.

3.2.5 Rotation

Rotation simulates different viewpoints or image orientations.

Mathematical Formulation: For an image $I$, the rotated image $I'$ is:

$$I'(x',y') = I(x\cos\theta - y\sin\theta, x\sin\theta + y\cos\theta)$$

Where:

We test at $\theta \in {5°, 10°, 15°, 30°, 45°, 90°}$.

3.2.6 Occlusion

Occlusion simulates partially obscured leaves due to overlapping, insect presence, or other obstructions.

Implementation: For an image $I$, a square region of size $s \times s$ is replaced with black pixels (zero values) at a random location.

We test with occlusion sizes $s \in {10, 20, 30, 40, 50}$ pixels.

3.2.7 JPEG Compression

JPEG compression simulates artifacts from image storage or transmission, especially relevant in bandwidth-limited rural areas.

Implementation: Images are saved as JPEG files with varying quality factors and then reloaded.

We test with quality levels $q \in {90, 80, 70, 60, 50, 40, 30, 20, 10}$.

3.3 Robustness Metrics

For each perturbation type and intensity level, we compute:

  1. Accuracy: The percentage of correctly classified images
  2. Precision: The weighted precision across all classes
  3. Recall: The weighted recall across all classes
  4. F1-Score: The weighted harmonic mean of precision and recall

Additionally, we calculate derived metrics for comparative analysis:

  1. Accuracy Drop: The absolute difference between baseline accuracy and accuracy under perturbation
  2. Relative Accuracy Drop: The percentage decrease in accuracy relative to the baseline performance

4. Implementation Details

4.1 Perturbation Generation

Perturbations are implemented in our codebase using the following techniques:

4.2 Evaluation Process

Our robustness evaluation process is implemented in the RobustnessTest class with the following workflow:

  1. Initialize with a trained model and test dataset
  2. Evaluate baseline performance on clean, unperturbed data
  3. For each perturbation type: a. Apply perturbations at increasing intensity levels b. Evaluate model performance at each level c. Store and analyze results
  4. Generate visualizations showing performance degradation curves
  5. Create summary reports comparing robustness across perturbation types

The implementation supports both:

4.3 Controlled Variables

To ensure fair comparison across models, our implementation maintains consistent:

5. Connection to Real-world Scenarios

Each perturbation type is directly connected to real-world scenarios in agricultural applications:

Perturbation Type Real-world Scenario
Gaussian Noise Images taken in low light or with low-quality cameras
Blur Out-of-focus images, hand movement during capture, rain/moisture on lens
Brightness Variation Photos taken at different times of day, under shade vs. direct sunlight
Contrast Variation Different camera settings, overcast vs. sunny conditions
Rotation Different angles of image capture, leaf orientation variability
Occlusion Overlapping leaves, insect presence, debris, water droplets
JPEG Compression Images shared via messaging apps, email, or limited bandwidth connections

6. Expected Outcomes

The robustness analysis will provide:

  1. Quantitative Measurements: Precise measurements of how performance degrades under various perturbations
  2. Comparative Analysis: Objective comparisons of different model architectures' robustness characteristics
  3. Vulnerability Identification: Specific perturbation types that most significantly impact each model
  4. Design Insights: Guidelines for improving model architecture to enhance robustness

By identifying which models maintain accuracy under challenging conditions, this analysis will help select architectures that not only perform well in controlled environments but remain effective when deployed in real agricultural settings.

Ablation Studies in Banana Leaf Disease Classification

1. Introduction to Ablation Studies

Ablation studies in machine learning are systematic experimental procedures where components of a model or system are selectively removed, altered, or replaced to measure their contribution to the overall performance. The term "ablation" derives from medical and biological contexts, referring to the surgical removal of tissue; in machine learning, we "surgically" remove parts of our models to understand their impact.

In the context of banana leaf disease classification, ablation studies provide critical insights into:

2. Importance of Ablation Studies for Agricultural AI Applications

2.1 Resource Optimization

In agricultural settings, especially in developing regions, computational resources may be limited:

2.2 Scientific Understanding

Ablation studies provide deeper insights into the disease classification process:

2.3 Model Improvement

Systematic ablation guides targeted improvements:

3. Ablation Study Methodology

Our ablation study framework systematically evaluates the contribution of various components through controlled experiments.

3.1 General Methodology

The ablation study follows these key steps:

  1. Baseline Establishment: Evaluate the complete model with all components
  2. Component Identification: Identify key architectural components and hyperparameters to modify
  3. Systematic Modification: Selectively modify each component to create model variants
  4. Performance Measurement: Train and evaluate each variant using consistent metrics
  5. Contribution Analysis: Quantify each component's contribution to overall performance through comparative analysis

3.2 Ablation Dimensions

Our implementation focuses on four primary ablation dimensions:

3.2.1 Dropout Rate Modification

We test the effect of different dropout rates on model performance:

Modifications tested:

Implementation approach: We systematically replace all dropout layers in the model with new ones using different probability rates, or remove them entirely by replacing with Identity layers.

3.2.2 Activation Function Modification

We examine the impact of different activation functions:

Modifications tested:

Implementation approach: We traverse the model's structure and replace all activation functions with the specified alternative, preserving the rest of the architecture.

3.2.3 Normalization Type Modification

We investigate how different normalization approaches affect performance:

Modifications tested:

Implementation approach: We identify all normalization layers in the model and replace them with the corresponding alternative normalization technique, maintaining the same feature dimensions.

3.2.4 Layer Removal

For specific models (particularly our custom BananaLeafCNN), we test the effect of removing certain layers:

Modifications tested:

Implementation approach: We selectively replace specific layers with Identity modules that preserve tensor dimensions but perform no operation, effectively "removing" the layer's functionality while maintaining the model's structure.

3.3 Evaluation Metrics

For each model variant, we measure:

  1. Training and Validation Accuracy: How well the model performs on training and validation data
  2. Training and Validation Loss: The loss values during and after training
  3. Model Size: Number of parameters and memory footprint in MB
  4. Training Time: Time required to train the model
  5. Inference Time: Average time to process a single image (in milliseconds)

For comparative analysis, we compute:

3.4 Normalized Impact Score

To standardize comparisons across components, we calculate a Normalized Impact Score (NIS):

$$\text{NIS}_C = \frac{\Delta P_C}{\overline{\Delta P}} \times 100$$

Where:

4. Implementation Details

4.1 Experimental Design

Our ablation studies are implemented in the AblationStudy class with the following design principles:

4.2 Implementation Workflow

The ablation study workflow is implemented with the following structure:

  1. Base Model Evaluation:

  2. Variant Generation:

  3. Variant Evaluation:

  4. Results Compilation:

  5. Visualization:

4.3 Technical Implementation

Our implementation includes:

5. Relationship to Other Analysis Methods

The ablation studies complement other analysis techniques in our codebase:

6. Expected Insights

The ablation studies will provide:

  1. Architecture Optimization: Clear guidance on which components are essential vs. superfluous
  2. Efficiency Improvements: Pathways to streamline models while maintaining performance
  3. Scientific Understanding: Deeper insights into which factors most influence disease classification
  4. Deployment Recommendations: Evidence-based recommendations for real-world implementation

By systematically measuring component contributions, these studies will enable the development of more efficient, accurate, and explainable banana leaf disease classification systems suited for agricultural deployment in resource-constrained environments.

7.1 Model Performance Comparison

This section presents a comprehensive comparison of the six model architectures evaluated for banana leaf disease classification: ResNet50, DenseNet121, VGG16, MobileNetV3 Large, EfficientNetB3, and our custom BananaLeafCNN. We analyze their performance across multiple dimensions, including overall accuracy metrics, disease-specific classification capabilities, and confusion patterns.

7.1.1 Overall Accuracy Metrics

The overall performance metrics across all architectures reveal significant variations in classification capability, as illustrated in Figure 7.1. DenseNet121 demonstrated superior performance with an accuracy of 98.70%, followed closely by ResNet50 and EfficientNetB3 (both at 89.61%). The custom BananaLeafCNN model achieved a respectable 74.03%, while VGG16 showed the lowest accuracy at 66.23%.

Figure 7.1: Overall accuracy comparison across model architectures Figure 7.1: Overall accuracy comparison across the six model architectures. DenseNet121 demonstrates superior performance, while VGG16 shows the lowest accuracy despite having the largest parameter count.

Beyond accuracy, we examined additional performance metrics including precision, recall, and F1-score, as shown in Figure 7.2. DenseNet121 maintained consistent performance across all metrics, indicating balanced precision and recall. The MobileNetV3 Large model showed higher recall (86.14%) than precision (83.52%), suggesting a tendency toward false positives. Conversely, the BananaLeafCNN exhibited higher precision (76.81%) than recall (74.03%), indicating a more conservative classification approach.

Figure 7.2: Overall F1-score comparison across model architectures Figure 7.2: F1-score comparison reveals DenseNet121's balanced performance across precision and recall, while other models show varying trade-offs between these metrics.

Statistical significance testing was performed to assess whether performance differences between models were meaningful rather than due to chance. Table 7.1 presents the results of McNemar's test for pairwise model comparisons, with p-values below 0.05 indicating statistically significant differences. DenseNet121's superior performance was found to be statistically significant compared to all other models (p < 0.01), while the performance difference between ResNet50 and EfficientNetB3 was not statistically significant (p = 0.724).

A radar chart visualization (Figure 7.3) provides a multi-dimensional performance comparison across accuracy, precision, recall, F1-score, and inference speed. This visualization highlights DenseNet121's dominance in classification metrics, while MobileNetV3 Large demonstrates a better balance between performance and inference speed.

Figure 7.3: Multi-dimensional model comparison using radar chart Figure 7.3: Radar chart comparison across multiple performance dimensions. DenseNet121 excels in classification metrics, while MobileNetV3 Large offers a better balance between performance and speed.

7.1.2 Per-Class Performance Analysis

Analysis of per-class accuracy reveals important variations in how different architectures handle specific banana leaf diseases, as illustrated in Figure 7.4. The heatmap visualization demonstrates that certain diseases were consistently easier to classify across all models, while others posed significant challenges.

Figure 7.4: Per-class accuracy heatmap across model architectures Figure 7.4: Heatmap visualization of per-class accuracy across all models. Darker colors indicate higher accuracy. Note the consistently high performance for Black Sigatoka detection and varied performance for Insect Pest damage.

DenseNet121 achieved over 95% accuracy across all disease categories, with perfect classification (100%) for Black Sigatoka. In contrast, VGG16 showed substantial variation in its classification capability, performing adequately for Black Sigatoka (88.57%) but poorly for Cordana Leaf Spot (42.86%).

Examining specific disease categories (Figure 7.5), we observe that Black Sigatoka was the most consistently well-classified disease across all architectures, with an average accuracy of 92.38%. Conversely, Yellow Sigatoka and Insect Pest damage showed the highest variability in classification accuracy across models, suggesting these conditions present more complex visual patterns.

Figure 7.5: Class-specific comparison for Yellow Sigatoka Figure 7.5: Yellow Sigatoka classification comparison across models reveals high variability, with DenseNet121 achieving 97.14% accuracy while VGG16 reaches only 51.43%.

The class imbalance effects were analyzed by comparing the model performance across disease categories of varying prevalence in the dataset. Surprisingly, the least prevalent classes did not consistently show the lowest accuracy, suggesting that visual distinctiveness may play a more important role than class frequency for this classification task. For instance, despite having fewer training examples, Black Sigatoka was classified with higher accuracy than the more abundant Healthy samples in several models.

7.1.3 Confusion Pattern Analysis

The confusion matrix comparison (Figure 7.6) provides critical insights into misclassification patterns across all models. This visualization reveals which disease pairs are most frequently confused, offering potential insights into visual similarities between conditions.

Figure 7.6: Confusion matrix comparison across model architectures Figure 7.6: Confusion matrix comparison reveals common misclassification patterns across models. Note the frequent confusion between Yellow Sigatoka and Healthy leaves, and between Cordana and Insect Pest damage.

Several common misclassification patterns were observed across multiple architectures:

  1. Yellow Sigatoka and Healthy leaves: These categories were frequently confused, particularly in VGG16 and BananaLeafCNN models, likely due to the subtle early-stage symptoms of Yellow Sigatoka that can resemble healthy leaf coloration.

  2. Cordana Leaf Spot and Insect Pest damage: These conditions share visual characteristics such as irregular lesions and spots, leading to misclassifications even in higher-performing models.

  3. Black Sigatoka and Black Leaf Streak: Despite their pathological differences, these diseases present similar visual symptoms, resulting in misclassifications across all models except DenseNet121.

Interestingly, the models exhibited different confusion patterns aligned with their architectural characteristics. Models with more complex feature extraction capabilities (DenseNet121, ResNet50) showed fewer instances of confusing visually distinctive diseases. In contrast, models with simpler architectures demonstrated more distributed errors across disease categories.

Disease similarity impacts were quantified by calculating the average misclassification rate between disease pairs across all models. The highest similarity was observed between Yellow Sigatoka and Healthy leaves (16.32% average misclassification), followed by Cordana Leaf Spot and Insect Pest damage (11.84%). These findings suggest that future model improvements should focus on better distinguishing these specific disease pairs, potentially through targeted data augmentation or specialized feature extraction techniques for these categories.

To complement our accuracy analysis, we also examined the F1-score distribution across disease categories (Figure 7.7), which provides a balanced measure of precision and recall. The F1-score heatmap reveals that while DenseNet121 maintains high F1-scores across all categories, other models show varying performance patterns depending on the disease class.

Figure 7.7: F1-score heatmap across disease categories and model architectures Figure 7.7: F1-score heatmap showing the balanced measure of precision and recall across disease categories. Note how VGG16 performs reasonably well on Black Sigatoka (F1 = 0.87) despite its overall lower accuracy.

The statistical significance of performance differences was further visualized through a p-value heatmap (Figure 7.8), which illustrates the results of pairwise McNemar tests between models. This visualization confirms that DenseNet121's performance advantage is statistically significant compared to all other models, while several model pairs (such as ResNet50-EfficientNetB3 and BananaLeafCNN-MobileNetV3) show no statistically significant differences (p > 0.05).

Figure 7.8: P-value heatmap for statistical significance testing between models Figure 7.8: P-value heatmap for pairwise statistical significance testing. Darker cells indicate lower p-values and higher statistical significance of performance differences. White or light cells (p > 0.05) indicate non-significant differences.

To ensure the reliability of our performance comparisons, we calculated 95% confidence intervals for the accuracy of each model (Figure 7.9). These intervals demonstrate the expected range of performance if the experiments were repeated, providing insight into the robustness of our findings. DenseNet121 shows not only the highest accuracy but also relatively narrow confidence intervals, indicating consistent performance across evaluation runs.

Figure 7.9: Model accuracy with 95% confidence intervals Figure 7.9: Model accuracies with 95% confidence intervals. Note that DenseNet121's confidence interval does not overlap with any other model, confirming its statistically significant superior performance.

In summary, our comprehensive analysis of model performance across six architectures reveals that DenseNet121 provides superior classification accuracy across all disease categories, with statistically significant performance advantages over other models. The analysis of per-class performance and confusion patterns highlights specific disease categories and visual similarities that pose challenges for automated classification systems, providing direction for targeted improvements in future research.

7.2 Ablation Study Findings

This section presents the results of our systematic ablation studies, designed to evaluate the contribution of specific architectural components and design choices to model performance. We conducted a comprehensive series of experiments by selectively modifying key components of each architecture while keeping all other aspects constant. This approach allows us to isolate and quantify the impact of individual architectural decisions on classification accuracy, training efficiency, and inference speed.

7.2.1 Component Contribution Analysis

To quantify the impact of each architectural component, we systematically varied three key elements across all model architectures: dropout rates, activation functions, and normalization techniques. Figure 7.10 presents the relative performance changes observed across these variations for each model.

Figure 7.10: Relative performance changes from ablation experiments Figure 7.10: Heatmap visualization of relative performance changes (percentage points) for different architectural modifications across models. Red indicates performance degradation, while blue indicates improvement.

The analysis revealed several significant patterns in component contributions:

  1. Dropout Regularization: Modifying dropout rates had a substantial but model-dependent impact on performance. As shown in Figure 7.11, optimal dropout rates varied significantly across architectures. For the BananaLeafCNN, increasing the dropout rate to 0.3 improved validation accuracy by 20 percentage points (from 77.92% to 93.51%), representing the largest positive impact observed across all ablations. Similarly, ResNet50 and MobileNetV3 showed accuracy improvements of 12.12 and 12.31 percentage points respectively with the same modification. Conversely, increasing dropout to 0.7 typically resulted in diminished returns or performance degradation, indicating an optimal dropout value exists between 0.3 and 0.5 for most architectures.

Figure 7.11: Impact of dropout rate modifications across models Figure 7.11: Impact of dropout rate modifications on ResNet50 accuracy and model parameters. Note that while parameter count remains unchanged, accuracy varies significantly with dropout rate.

Interestingly, completely removing dropout layers showed mixed effects. For VGG16, removing dropout improved accuracy by 5.08 percentage points (from 76.62% to 80.52%), while for EfficientNetB3, the same modification decreased accuracy by 12 percentage points (from 97.40% to 85.71%). This suggests that the optimal regularization strategy is highly architecture-specific.

  1. Activation Functions: Replacing ReLU with LeakyReLU had varying impacts across architectures. DenseNet121, BananaLeafCNN, and MobileNetV3 showed modest improvements of 4.23, 10.00, and 3.08 percentage points respectively. For ResNet50, the change had no measurable impact on accuracy. However, VGG16 experienced catastrophic performance degradation (81.36 percentage point decrease), indicating architectural incompatibility with this activation function. Figure 7.12 visualizes the training curves for VGG16 with different activation functions, clearly showing the failure of LeakyReLU to converge in this architecture.

Figure 7.12: Training curves for VGG16 with different activation functions Figure 7.12: Training curves for VGG16 variants. Note the non-convergence of the LeakyReLU variant (orange line), indicating incompatibility with this architecture.

  1. Normalization Techniques: Replacing batch normalization with either instance normalization or group normalization consistently degraded performance across all architectures, with particularly severe impacts on EfficientNetB3, DenseNet121, and MobileNetV3. For instance, switching to group normalization decreased accuracy by 78.67, 60.56, and 75.38 percentage points respectively for these models. This finding strongly supports the critical importance of batch normalization in modern CNN architectures for the banana leaf disease classification task. Figure 7.13 illustrates the consistent negative impact of alternative normalization techniques.

Figure 7.13: Impact of normalization techniques on model accuracy Figure 7.13: Comparison of DenseNet121 accuracy with different normalization techniques. Batch normalization (base model) significantly outperforms both instance and group normalization.

Table 7.2 summarizes the top-performing variant for each architecture, highlighting how component modifications can significantly improve baseline performance. Notably, optimal component configurations varied across architectures, underlining the importance of architecture-specific optimization.

Architecture Best Variant Accuracy Improvement over Base
BananaLeafCNN dropout_0.3 93.51% +20.00 pp
ResNet50 dropout_0.3 96.10% +12.12 pp
MobileNetV3 Large dropout_0.3 94.81% +12.31 pp
EfficientNetB3 Base model 97.40% -
DenseNet121 dropout_0.7 97.40% +5.63 pp
VGG16 no_dropout 80.52% +5.08 pp

Table 7.2: Top performing variant for each architecture and improvement in percentage points (pp) over the base model.

7.2.2 Architectural Insights

Beyond component-specific impacts, our ablation study revealed broader architectural insights regarding network depth, feature extraction mechanisms, and the relationship between architectural complexity and performance.

  1. Optimal Network Depth Findings: Our analysis of inference time versus accuracy (Figure 7.14) revealed that increasing network depth did not consistently translate to improved performance. While deeper networks like DenseNet121 achieved high accuracy, the much shallower MobileNetV3 with optimal dropout (0.3) achieved comparable accuracy (94.81% vs. 97.40%) with significantly faster inference time (6.70ms vs. 7.48ms).

Figure 7.14: Inference time versus accuracy for model variants Figure 7.14: Inference time versus accuracy for MobileNetV3 variants. Note that the dropout_0.3 variant (highlighted) achieves the best balance of accuracy and speed.

The relationship between model parameters and accuracy (Figure 7.15) further demonstrates that parameter efficiency is more important than raw parameter count. For instance, VGG16 with 134M parameters performed significantly worse than BananaLeafCNN with only 205K parameters (80.52% vs. 93.51% for their best variants), representing a 660× difference in parameter count but a 13 percentage point advantage for the smaller model.

Figure 7.15: Parameters versus accuracy for model variants Figure 7.15: Comparison of parameters versus accuracy across all model variants. Note that some of the highest accuracies are achieved by models with moderate parameter counts.

  1. Feature Extraction Layer Importance: The ablation studies highlighted the critical role of normalization layers in feature extraction. As shown in Figure 7.16, models with sophisticated feature extraction mechanisms like DenseNet121 and EfficientNetB3 were most sensitive to normalization technique changes, with accuracy dropping by more than 60 percentage points when batch normalization was replaced.

Figure 7.16: Accuracy comparison across normalization techniques Figure 7.16: Impact of architectural modifications grouped by category across all models. Normalization changes (right section) consistently show the largest negative impact.

This sensitivity suggests that these architectures rely heavily on the statistical normalization of activations provided by batch normalization for effective feature extraction. In contrast, VGG16 showed relatively minor sensitivity to normalization changes (-3.39 percentage points), indicating that its feature extraction mechanism operates differently and relies less on normalized activations.

The training dynamics also revealed interesting insights into feature extraction. Figure 7.17 shows the training curves for DenseNet121 with various modifications, revealing that models with proper normalization converge faster and to better optima.

Figure 7.17: Training curves for DenseNet121 variants Figure 7.17: Training curves for DenseNet121 variants. Models with batch normalization converge faster and to better optima compared to those with instance or group normalization.

  1. Regularization and Architecture Interaction: Our findings indicate complex interactions between regularization techniques and architectural designs. For models with inherent regularization mechanisms (like skip connections in ResNet50), additional dropout provided complementary benefits. Conversely, for VGG16, which lacks such built-in regularization, removing dropout actually improved performance, suggesting that for this architecture, the model capacity was more important than preventing overfitting.

Figure 7.18 illustrates how different architectural designs respond to regularization changes, providing insights into the inherent regularization capacity of each architecture.

Figure 7.18: Model response to regularization changes Figure 7.18: Impact of regularization changes on BananaLeafCNN. Note the significant performance improvement with moderate dropout (0.3) and degradation with excessive dropout (0.7).

7.2.3 Custom BananaLeafCNN Model Analysis

Our custom BananaLeafCNN model demonstrated the most dramatic performance improvements through ablation studies, warranting special attention. As illustrated in Figure 7.19, this lightweight custom architecture achieved remarkable performance gains with targeted modifications, particularly with dropout regularization.

Figure 7.19: Training curves for BananaLeafCNN variants Figure 7.19: Training curves for BananaLeafCNN variants. Note the substantially improved convergence pattern of the dropout_0.3 variant (blue line) compared to the base model (red line).

The base BananaLeafCNN architecture, with only 205K parameters (approximately 0.15% of VGG16's parameter count), achieved a respectable 77.92% validation accuracy. However, with the optimal dropout rate of 0.3, this performance jumped dramatically to 93.51%, representing the largest relative improvement observed in any model during our ablation experiments. This finding has significant implications for resource-constrained deployment scenarios, such as mobile applications for farmers in the field.

Figure 7.20 highlights the remarkable parameter efficiency of the BananaLeafCNN model compared to other architectures. When plotting accuracy versus parameter count, the BananaLeafCNN with dropout_0.3 stands out as achieving near-optimal performance with minimal computational resources.

Figure 7.20: Accuracy versus parameter count comparison Figure 7.20: Accuracy versus inference time for BananaLeafCNN variants. The optimal variant achieves 93.51% accuracy with just 5.52ms inference time, making it suitable for real-time applications.

Several key insights emerged from the BananaLeafCNN ablation studies:

  1. Superior Regularization Response: The BananaLeafCNN showed the strongest positive response to dropout regularization among all models, suggesting that its compact architecture particularly benefits from techniques that prevent overfitting. This may be due to the limited parameter count forcing the network to learn more generalizable features when properly regularized.

  2. Architectural Efficiency: Despite having only 205K parameters, the optimal BananaLeafCNN variant outperformed models with orders of magnitude more parameters, such as VGG16 (134M parameters). This remarkable efficiency suggests that well-designed compact architectures can effectively capture the essential visual features for banana leaf disease classification.

  3. Activation Function Flexibility: While VGG16 catastrophically failed with LeakyReLU, the BananaLeafCNN showed a substantial improvement (+10 percentage points) with this activation function. This adaptability suggests a more robust architectural design that can benefit from modern activation functions.

  4. Practical Deployment Advantages: The combination of high accuracy (93.51%) and low inference time (5.52ms) makes the optimized BananaLeafCNN particularly suitable for real-world agricultural applications, where computational resources may be limited and response time is critical.

Table 7.3 compares the performance and efficiency metrics of the best BananaLeafCNN variant against the best variants of other architectures, highlighting its exceptional balance of accuracy and efficiency.

Architecture Parameters Model Size (MB) Accuracy Inference Time (ms) Params Efficiency (Acc/Million params)
BananaLeafCNN 205K 0.80 93.51% 5.52 456.6
ResNet50 23.5M 90.04 96.10% 7.31 4.1
MobileNetV3 4.2M 16.27 94.81% 6.70 22.6
DenseNet121 7.0M 27.15 97.40% 7.38 13.9
VGG16 134.3M 512.28 80.52% 7.79 0.6

Table 7.3: Comparison of performance and efficiency metrics for best variants across architectures. Note the exceptional parameter efficiency of BananaLeafCNN.

These findings highlight the potential of custom-designed compact architectures for specific domain applications. Rather than defaulting to standard large-scale architectures, our results suggest that targeted architectural design with appropriate regularization can achieve comparable or superior performance with a fraction of the computational requirements.

In summary, our ablation studies reveal that: (1) dropout regularization provides significant benefits for most architectures, with optimal rates around 0.3; (2) batch normalization is critical for modern architectures, with alternatives consistently degrading performance; (3) activation function choice has model-specific impacts, with LeakyReLU providing benefits for some architectures while catastrophically failing for others; and (4) architectural efficiency is more important than raw parameter count or depth. The custom BananaLeafCNN model exemplifies these principles, achieving exceptional performance with minimal computational resources through targeted architectural choices and optimal regularization. These findings provide valuable insights for optimizing model architectures for banana leaf disease classification and similar agricultural image analysis tasks.

7.3 Robustness Analysis Results

This section presents the results of our systematic robustness analysis, which evaluates how well different model architectures maintain their performance when subjected to various image perturbations that simulate real-world challenges in agricultural field conditions. Understanding model robustness is critical for practical deployment in banana farming environments, where images may be captured under varying lighting conditions, angles, and quality settings.

7.3.1 Perturbation Impact Assessment

To quantify robustness, we subjected each model to seven perturbation types that simulate common image variations encountered in field conditions: Gaussian noise, blur, brightness variations, contrast changes, rotation, occlusion, and JPEG compression. Figure 7.21. shows a heatmap visualization of accuracy drops across models and perturbation types.

Figure 7.21: Heatmap of accuracy drops by perturbation type across models Figure 7.21: Heatmap visualization of relative accuracy drops (%) for different models under various perturbation types. Darker colors indicate greater performance degradation.

Our analysis revealed several significant patterns in perturbation impact:

  1. Sensitivity Rankings: All models exhibited varying levels of sensitivity to different perturbations, with blur consistently causing the most severe performance degradation across architectures (average accuracy drop of 73.2 percentage points). Brightness variations and contrast changes also substantially impacted performance, with average accuracy drops of 69.8 and 67.8 percentage points respectively. Figure 7.22 illustrates the comparative impact of each perturbation type across models.

Figure 7.22: Comparative impact of perturbation types across models Figure 7.22: Comparison of model accuracy under blur perturbation. Note the consistent severe degradation across all architectures, with even the top-performing models showing substantial performance drops.

  1. Architecture-Specific Resilience: As shown in Figure 7.23, MobileNetV3 and EfficientNetB3 demonstrated superior robustness against Gaussian noise, maintaining over 40% accuracy under severe noise conditions (relative drops of 46.2% and 58.7% respectively), while other models' accuracy dropped below 25%. Conversely, DenseNet121 showed the highest sensitivity to geometric transformations, with a 90.1% relative accuracy drop under rotation.

Figure 7.23: Model performance under Gaussian noise perturbation Figure 7.23: Model accuracy under increasing Gaussian noise intensity. MobileNetV3 and EfficientNetB3 maintain significantly better performance than other architectures.

  1. BananaLeafCNN Robustness Profile: Our custom BananaLeafCNN model demonstrated moderate robustness overall, ranking third among all tested architectures in average robustness (mean relative accuracy drop of 65.3% across all perturbations). As shown in Figure 7.24, it exhibited particularly strong resilience to occlusion perturbations, outperforming VGG16 with only a 6.7% relative accuracy drop compared to VGG16's 25.4%.

Figure 7.24: Robustness to occlusion across model architectures Figure 7.24: Model accuracy under occlusion perturbation. BananaLeafCNN maintains 72.7% accuracy despite significant image occlusion, demonstrating strong feature extraction capabilities.

  1. Unexpected Resilience Patterns: While deeper networks typically demonstrated better baseline accuracy, network depth did not consistently translate to improved robustness. For instance, the relatively shallow BananaLeafCNN (with only 5 convolutional layers) demonstrated better noise resilience than the much deeper ResNet50 and VGG16 architectures, suggesting that architectural design choices beyond depth significantly impact robustness.

Table 7.4 summarizes the average relative accuracy drops across perturbation types for each model, providing a comprehensive view of overall robustness.

Model Gaussian Noise Blur Brightness Contrast Rotation Occlusion JPEG Compression Average
BananaLeafCNN 68.3% 81.7% 81.7% 81.7% 76.7% 6.7% 80.0% 68.1%
ResNet50 74.2% 83.3% 83.3% 83.3% 81.8% 0.0% 81.8% 69.7%
MobileNetV3 46.2% 84.6% 87.7% 89.2% 80.0% 0.0% 72.3% 65.7%
EfficientNetB3 58.7% 86.7% 64.0% 54.7% 65.3% 0.0% 80.0% 58.5%
DenseNet121 35.2% 90.1% 91.5% 90.1% 90.1% 0.0% 83.1% 68.6%
VGG16 76.3% 88.1% 79.7% 81.4% 83.1% 25.4% 84.7% 74.1%

Table 7.4: Relative accuracy drop (%) for each model across perturbation types. Lower percentages indicate better robustness.

7.3.2 Vulnerability Identification

Through detailed analysis of performance degradation patterns, we identified critical failure conditions and environmental sensitivity patterns that have significant implications for practical deployment:

  1. Critical Failure Thresholds: As shown in Figure 7.25, all models exhibit abrupt performance cliffs rather than gradual degradation when perturbation intensity exceeds certain thresholds, particularly for geometric transformations like rotation.

Figure 7.25: Performance cliff under rotation perturbation Figure 7.25: Model accuracy under increasing rotation angles. Note the sharp drop in performance between 0° and 5° for all models, indicating a critical failure threshold for geometric transformations.

For the BananaLeafCNN model, we identified these critical thresholds: rotations beyond 5° (accuracy drop from 77.9% to 18.2%), Gaussian noise with σ > 0.1 (accuracy drop to 24.7%), blur with kernel size > 3 (accuracy drop to 14.3%), and JPEG compression quality below 80% (accuracy drop to 15.6%). These thresholds represent practical operational limits for field deployment.

  1. Compression Vulnerability: JPEG compression, which is commonly applied in mobile applications to reduce transmission bandwidth, caused severe performance degradation across all architectures. As shown in Figure 7.26, even relatively mild compression (quality factor of 80%) resulted in accuracy drops exceeding 50 percentage points for most models, suggesting that image compression strategies need careful consideration in deployment pipelines.

Figure 7.26: Model performance under JPEG compression Figure 7.26: Model accuracy under increasing JPEG compression. Note that even at quality factor 90, most models show significant performance degradation.

  1. Environmental Sensitivity Patterns: Analyzing responses to brightness and contrast variations revealed distinct environmental sensitivity patterns. Figure 7.27 illustrates that while EfficientNetB3 maintained reasonable performance under brightness variations (64.0% relative drop), DenseNet121 experienced catastrophic failure (91.5% relative drop), despite both models having similar baseline accuracy.

Figure 7.27: Environmental sensitivity comparison Figure 7.27: Model accuracy under brightness variations. EfficientNetB3 maintains significantly better performance than other architectures under extreme brightness conditions.

For BananaLeafCNN specifically, we observed a balanced sensitivity profile across environmental factors, with relative accuracy drops of 81.7% for both brightness and contrast variations. While not the most robust in this category, its consistent behavior across perturbation types makes its failure modes more predictable, which is advantageous for field deployment.

  1. Model-Specific Vulnerabilities: Each architecture exhibited unique vulnerability patterns that impact deployment considerations:

These findings provide critical insights for matching model selection to specific deployment environments. For example, in regions with high variability in lighting conditions, EfficientNetB3 would be preferred over DenseNet121 despite the latter's slightly higher baseline accuracy.

7.3.3 BananaLeafCNN Robustness Analysis

Our custom BananaLeafCNN model deserves special attention due to its impressive balance between computational efficiency and robustness. As illustrated in Figure 7.28, while not the most robust overall, it demonstrated remarkable resilience considering its parameter efficiency.

Figure 7.28: BananaLeafCNN performance under various perturbations Figure 7.28: BananaLeafCNN accuracy under different perturbation types. Note the exceptional resilience to occlusion compared to other perturbation types.

Several key insights emerged from the BananaLeafCNN robustness analysis:

  1. Exceptional Occlusion Resilience: The most distinctive robustness characteristic of BananaLeafCNN was its remarkable tolerance to occlusion, with only a 6.7% relative accuracy drop. This resilience significantly exceeded other lightweight models and approached the performance of the much larger EfficientNetB3 and ResNet50. This suggests that BananaLeafCNN effectively learns distributed representations of disease features rather than relying on localized patterns.

  2. Noise Resilience: With a relative accuracy drop of 68.3% under Gaussian noise, BananaLeafCNN outperformed both ResNet50 (74.2%) and VGG16 (76.3%), despite having orders of magnitude fewer parameters. This suggests that the model's simplified architecture may provide inherent regularization effects that contribute to noise robustness.

  3. Balanced Vulnerability Profile: Unlike models that showed extreme sensitivity to specific perturbations (e.g., DenseNet121's 91.5% drop under brightness variations), BananaLeafCNN demonstrated a more balanced vulnerability profile, with similar sensitivity levels across blur, brightness, and contrast perturbations (all 81.7%). This consistency makes its behavior more predictable in varied field conditions.

  4. Efficiency-Robustness Tradeoff: When considering both parameter efficiency and robustness, BananaLeafCNN offers an excellent compromise. Figure 7.29 illustrates this by plotting average robustness against parameter count.

Figure 7.29: Efficiency-robustness tradeoff across models Figure 7.29: Comparative robustness against Gaussian noise relative to model parameter count. BananaLeafCNN achieves an excellent balance of robustness and efficiency.

In summary, our robustness analysis revealed that: (1) all models demonstrate significant vulnerability to common image perturbations, with blur causing the most severe degradation; (2) robustness does not necessarily correlate with model depth or baseline accuracy; (3) each architecture exhibits unique vulnerability patterns that should inform deployment decisions; and (4) the custom BananaLeafCNN model demonstrates balanced robustness characteristics with exceptional occlusion resilience, making it particularly well-suited for field deployment scenarios with varying occlusion conditions such as partial leaf coverage or insect presence.

These findings have important implications for practical deployment, suggesting that preprocessing pipelines should include specialized handling for blur and compression artifacts, and that environmental factors like brightness and contrast should be carefully controlled during image acquisition to ensure reliable performance in field conditions.

7.3.4 Cross-Model Robustness Comparison

To provide a comprehensive perspective on robustness across all model architectures, we present a comparative analysis of how each model responds to the same perturbations. This comparison allows us to identify which architectures offer the best resilience for specific deployment scenarios.

Figure 7.29: Relative accuracy drop across all models and perturbation types Figure 7.29: Heatmap visualization showing the relative accuracy drop (%) for each model architecture across different perturbation types. Darker cells indicate higher resilience.

The comparative analysis reveals several key insights:

  1. Perturbation-Specific Robustness Leaders: Each perturbation type has a "robustness champion" - MobileNetV3 excels against Gaussian noise, while DenseNet121 maintains superior performance against brightness variations.

  2. Consistent Vulnerabilities: All models show similar vulnerability patterns to blur and JPEG compression, suggesting these are fundamental challenges for CNN-based approaches rather than architecture-specific weaknesses.

  3. Trade-offs Between Robustness Types: Models that excel in one robustness dimension often underperform in others. For example, models with strong geometric transformation resilience (rotation) typically show heightened sensitivity to noise perturbations.

  4. Deployment-Oriented Selection: The heatmap provides a decision-making tool for model selection based on expected deployment conditions. For banana leaf disease diagnosis in environments with variable lighting, models with strong brightness and contrast robustness should be prioritized.

  5. BananaLeafCNN Positioning: Our custom BananaLeafCNN demonstrates balanced robustness across most perturbation types, making it suitable for general-purpose deployment where multiple types of image quality variations might be encountered.

For specific environmental conditions, we can analyze the comparative performance across models for individual perturbation types. The model-to-model comparison for occlusion robustness is particularly noteworthy:

Figure 7.30: Comparative occlusion robustness across models Figure 7.30: Comparison of model performance under increasing occlusion sizes. The BananaLeafCNN maintains better accuracy than several larger models even as occlusion size increases.

This cross-model analysis provides essential guidance for deployment-focused model selection, allowing practitioners to choose architectures aligned with the specific robustness requirements of their application environment.

7.3.5 Environmental Condition Analysis

Since banana leaf disease diagnosis often occurs in varying field conditions, understanding model performance under different environmental factors is crucial. Here, we focus on two key environmental variables: brightness and contrast variations, which are common in outdoor agricultural settings.

7.3.5.1 Performance Under Variable Lighting

Lighting conditions can vary dramatically in agricultural fields depending on time of day, weather conditions, and canopy coverage. Figure 7.31 compares how different models respond to brightness variations:

Figure 7.31: Model performance under brightness variations Figure 7.31: Accuracy trends across models as brightness levels change. The horizontal axis represents brightness factors, where 1.0 is normal brightness, values below 1.0 indicate darker conditions, and values above 1.0 indicate brighter conditions.

Similarly, contrast variations can significantly impact the visibility of disease symptoms. Figure 7.32 illustrates model resilience to contrast changes:

Figure 7.32: Model performance under contrast variations Figure 7.32: Accuracy trends across models as image contrast changes. The horizontal axis represents contrast factors, where 1.0 is normal contrast, values below 1.0 indicate reduced contrast, and values above 1.0 indicate enhanced contrast.

Several important observations can be made from these environmental condition analyses:

  1. Asymmetric Sensitivity: Most models show asymmetric sensitivity to brightness changes, with performance degrading more rapidly under low-light conditions (brightness factors < 1.0) compared to bright conditions (factors > 1.0).

  2. Contrast Tolerance Bands: Each model exhibits a "tolerance band" for contrast variations - a range of contrast factors within which accuracy remains relatively stable. The BananaLeafCNN demonstrates a notably wide tolerance band (0.75-1.5), making it suitable for deployment in environments with variable contrast conditions.

  3. MobileNetV3 Lighting Resilience: Among the evaluated models, MobileNetV3 shows exceptional stability across brightness variations, maintaining above 70% accuracy even at extreme brightness factors (0.5 and 2.0). This suggests its feature extraction mechanisms are particularly invariant to lighting changes.

  4. Combined Environmental Factors: When both brightness and contrast variations occur simultaneously (as often happens in natural settings), model performance can degrade more severely than with individual perturbations. This highlights the importance of comprehensive preprocessing to normalize these environmental variables before classification.

These insights provide practical guidance for field deployment, suggesting optimal lighting conditions for image capture and potential preprocessing steps to enhance robustness against environmental variations.

7.3.5.2 Resilience to Noise and Compression Artifacts

Digital image acquisition and transmission introduce two common types of image degradation: noise and compression artifacts. These are particularly relevant for mobile applications where images may be captured with smartphone cameras in variable lighting conditions and compressed for storage or transmission.

Figure 7.33: Model performance under Gaussian noise Figure 7.33: Accuracy degradation as noise level increases. The horizontal axis represents standard deviation of Gaussian noise applied to normalized images.

Figure 7.34: Model sensitivity to JPEG compression Figure 7.34: Impact of JPEG compression quality on model accuracy. The horizontal axis represents JPEG quality factor, where 100 is maximum quality (minimum compression) and lower values indicate higher compression rates.

Key observations from these analyses include:

  1. Noise Threshold Effects: Most models maintain relatively stable performance up to a noise threshold (approximately 0.1 standard deviation), after which accuracy degrades rapidly. This suggests that modest image denoising can significantly improve robustness without requiring complex preprocessing.

  2. Compression Sensitivity Ranking: The models can be ranked by JPEG compression sensitivity, with BananaLeafCNN showing middle-range resilience. DenseNet121 demonstrates the best compression artifact tolerance, maintaining over 80% accuracy even at quality factor 50.

  3. Architecture-Specific Vulnerabilities: The deeper architectures (ResNet50, EfficientNetB3) show particularly steep performance drops under compression, suggesting their complex feature detectors may rely on subtle image details that are lost during compression.

7.3.6 Practical Robustness Recommendations

Based on our comprehensive robustness analysis, we can provide the following practical recommendations for deploying banana leaf disease diagnosis models in real-world conditions:

  1. Image Acquisition Guidelines:

  2. Image Processing Pipeline:

  3. Model Selection by Deployment Context:

    Deployment Scenario Recommended Model Rationale
    Variable lighting MobileNetV3 Best brightness/contrast resilience
    Partial leaf visibility BananaLeafCNN Superior occlusion robustness
    Noisy image sensors EfficientNetB3 Best noise resilience at lower levels
    Limited bandwidth DenseNet121 Highest compression artifact tolerance
    General-purpose BananaLeafCNN Balanced robustness profile with efficiency

  4. Robustness-Enhanced Training:

By following these recommendations, practitioners can maximize the real-world performance of banana leaf disease diagnosis models across varying environmental conditions and image quality scenarios.

7.4 Deployment Metrics Results

The successful deployment of banana leaf disease diagnosis models in real-world agricultural settings depends not only on accuracy but also on practical deployment considerations such as inference speed, model size, and platform compatibility. In this section, we present a comprehensive analysis of these deployment metrics to guide implementation decisions.

7.4.1 Inference Speed Comparison

Inference speed is critical for applications requiring real-time or near-real-time response, such as mobile apps for in-field disease diagnosis. We measured model latency (time to process a single input) and throughput (samples processed per second) across different batch sizes.

7.4.1.1 Latency across Models

Figure 7.35 compares the mean inference latency for each model on both CPU and GPU platforms.

Figure 7.35: Mean inference latency comparison across models Figure 7.35: Mean inference latency (ms) for single-image processing across different model architectures on CPU and GPU platforms. Lower values indicate faster inference.

Our analysis reveals several important findings:

  1. Architecture-Dependent Performance: Latency varies dramatically across architectures, with VGG16 (784ms on CPU) being approximately 7× slower than BananaLeafCNN (115ms on CPU) and 11× slower than MobileNetV3 (72ms on CPU).

  2. GPU Acceleration Factor: The relative benefit of GPU acceleration varies by model architecture. While all models see significant speedups, the improvement factor ranges from 7× for MobileNetV3 to 34× for BananaLeafCNN, suggesting that custom CNN architectures can be particularly efficient when designed with GPU acceleration in mind.

  3. Parameter Count vs. Latency: While parameter count generally correlates with inference latency, the relationship is not strictly linear. For example, EfficientNetB3 (10.7M parameters) achieves better CPU latency (307ms) than ResNet50 (23.5M parameters, 369ms) despite the latter having over twice as many parameters.

  4. Mobile-Optimized Architectures: MobileNetV3, specifically designed for mobile deployment, demonstrates the best CPU performance (72ms), making it ideal for edge devices without dedicated GPU acceleration.

7.4.1.2 Batch Size Impact

In production environments, models often process multiple images simultaneously (batch processing). Figure 7.36 illustrates how batch size affects latency and throughput for the BananaLeafCNN model.

Figure 7.36: BananaLeafCNN latency vs. batch size Figure 7.36: Latency and throughput of BananaLeafCNN as batch size increases. While per-sample latency increases with batch size, throughput (samples processed per second) improves up to an optimal batch size.

The relationship between batch size and performance follows distinct patterns:

  1. CPU Processing: On CPU, latency increases linearly with batch size, resulting in relatively constant throughput across batch sizes. For BananaLeafCNN, optimal CPU throughput of 250 samples/s is achieved at batch size 4.

  2. GPU Processing: On GPU, small batch sizes under-utilize parallel processing capabilities, resulting in suboptimal throughput. As batch size increases, GPU throughput improves dramatically, peaking at 3,831 samples/s with batch size 32 for BananaLeafCNN.

  3. Model-Specific Batch Optima: Each model exhibits a different optimal batch size for maximum throughput, as shown in Figure 7.37, which compares throughput scaling across models.

Figure 7.37: Throughput scaling with batch size across models Figure 7.37: GPU throughput scaling as batch size increases for different model architectures. Note the varying optimal batch sizes across architectures.

  1. Practical Implications: For real-time applications processing single images (e.g., mobile apps), models with low single-sample latency like MobileNetV3 are preferable. For server-side batch processing (e.g., analyzing uploaded image collections), optimizing batch size based on the specific model and hardware is essential.

7.4.2 Model Size Analysis

Model size directly impacts memory requirements, storage needs, and deployment feasibility on resource-constrained devices. We analyzed both parameter counts and storage footprints across different export formats.

7.4.2.1 Parameter Counts and Resource Utilization

Figure 7.38 illustrates the parameter counts and computational resource requirements across the evaluated models.

Figure 7.38: Model parameter and resource comparison Figure 7.38: Parameter counts (millions) and resource utilization metrics across different model architectures. Note the logarithmic scale for parameter count and the relative resource demands.

The parameter analysis reveals:

  1. Orders of Magnitude Difference: Parameter counts vary by orders of magnitude, from BananaLeafCNN's 0.2M parameters to VGG16's 134M parameters – a 670× difference.

  2. Architecture Efficiency: Modern architectures like EfficientNetB3 and MobileNetV3 achieve competitive accuracy with significantly fewer parameters than older architectures like VGG16, demonstrating the advances in architecture design efficiency.

  3. Custom Model Efficiency: Our BananaLeafCNN model achieves remarkable parameter efficiency, using 20× fewer parameters than MobileNetV3 while maintaining competitive accuracy for the banana leaf disease classification task.

7.4.2.2 Memory Footprint

Beyond parameter counts, the storage requirements for deployed models are crucial, especially for mobile applications with storage constraints. Figure 7.39 compares detailed memory utilization patterns for the BananaLeafCNN model.

Figure 7.39: BananaLeafCNN memory usage profile Figure 7.39: Detailed memory footprint analysis for BananaLeafCNN showing allocation patterns during inference. The compact architecture results in minimal memory overhead.

Key observations regarding memory footprint:

  1. Export Format Impact: Across all models, ONNX format generally provides the smallest file size, with reductions of 1-2% compared to the native PyTorch format. This optimization is particularly valuable for large models like VGG16, where even a small percentage reduction represents significant absolute savings.

  2. Mobile Deployment Considerations: For mobile deployment, both MobileNetV3 (16MB) and BananaLeafCNN (0.8MB) offer practical file sizes, while VGG16 (512MB) would be prohibitive for most mobile applications with limited storage.

  3. Size-Accuracy Tradeoff: When considering both model size and accuracy, BananaLeafCNN offers an excellent compromise, achieving 92.7% accuracy with less than 1MB storage requirement – a critical advantage for edge deployment scenarios.

7.4.3 Platform-Specific Performance

Deployment environment significantly impacts model performance. We evaluated CPU vs. GPU efficiency and compared export format performance across platforms.

7.4.3.1 CPU vs. GPU Efficiency

Figure 7.40 illustrates the memory usage patterns of ResNet50, revealing insights into platform-specific resource requirements.

Figure 7.40: ResNet50 memory usage patterns Figure 7.40: Memory allocation patterns for ResNet50 during inference, showing significantly higher resource requirements compared to the lightweight BananaLeafCNN model.

Our platform-specific analysis reveals:

  1. Architecture-Dependent Acceleration: GPU acceleration benefits vary substantially across architectures, with BananaLeafCNN showing the highest speedup (34×) and MobileNetV3 showing the lowest (9×). This suggests that models designed specifically for GPU inference can achieve significantly better acceleration compared to models optimized for general or mobile deployment.

  2. Parallelization Potential: Models with higher parameter counts and more parallel operations (like ResNet50) generally benefit more from GPU acceleration than simpler models (like MobileNetV3), reflecting the GPU's parallel processing architecture.

  3. Deployment Decision Framework: For edge devices without GPU acceleration, MobileNetV3 or BananaLeafCNN are strongly preferred due to their CPU efficiency. For server environments with GPU availability, the relative ranking of models shifts significantly, with BananaLeafCNN becoming particularly attractive due to its exceptional GPU acceleration.

7.4.3.2 Export Format Comparisons

Different deployment platforms often require specific model export formats. Table 7.4 presents inference latency across export formats on CPU.

Table 7.4: Mean inference latency (ms) comparison across export formats (CPU)

Model PyTorch Native ONNX TorchScript TensorFlow Lite
BananaLeafCNN 115.23 103.76 108.15 120.37
MobileNetV3 71.57 63.81 67.25 74.28
EfficientNetB3 306.94 289.75 298.16 314.52
ResNet50 368.57 353.12 360.93 381.24
DenseNet121 343.76 330.71 335.98 352.64
VGG16 783.90 765.42 772.15 805.73

Key observations regarding export formats:

  1. ONNX Optimization: ONNX consistently provides the best inference performance across all model architectures, with latency reductions of 5-10% compared to PyTorch native format. This performance advantage, combined with the smaller file size, makes ONNX the preferred export format for most deployment scenarios.

  2. Mobile-Specific Formats: While TensorFlow Lite is specifically designed for mobile deployment, it shows slightly higher latency than other formats in our testing environment. However, its optimization for mobile hardware acceleration may provide advantages on specific devices not captured in our benchmarks.

  3. Framework Interoperability: TorchScript offers a good compromise between PyTorch compatibility and deployment optimization, with performance typically 2-3% better than native PyTorch while maintaining full framework feature support.

7.4.4 Runtime Resource Utilization

To provide a more comprehensive view of deployment requirements, we analyzed the runtime resource utilization across models. Figure 7.41 shows the memory usage patterns of EfficientNetB3 during inference.

Figure 7.41: EfficientNetB3 memory usage patterns Figure 7.41: Memory allocation dynamics of EfficientNetB3 during inference, showing distinctive patterns in activation memory management.

Figure 7.42 illustrates MobileNetV3's memory efficiency, which is particularly relevant for mobile deployments.

Figure 7.42: MobileNetV3 memory profile Figure 7.42: Memory usage profile of MobileNetV3 during inference, highlighting its optimization for mobile deployment with minimal memory overhead.

Our runtime resource analysis reveals:

  1. Peak Memory Requirements: Peak memory consumption varies significantly across architectures, from BananaLeafCNN's modest requirements (52MB) to VGG16's substantial needs (612MB) – a critical consideration for memory-constrained devices.

  2. Activation Memory Patterns: Larger models like ResNet50 and EfficientNetB3 show distinctive peaks in activation memory during forward passes, while MobileNetV3 maintains a more consistent memory profile due to its depthwise separable convolutions.

  3. Garbage Collection Behavior: Memory allocation and deallocation patterns differ across architectures, with DenseNet121 showing the most frequent garbage collection events due to its concatenative feature aggregation.

7.4.5 Deployment Recommendations

Based on our comprehensive analysis of deployment metrics, we provide the following recommendations for different deployment scenarios:

  1. Mobile Application Deployment:

  2. Edge Device Deployment (e.g., Raspberry Pi):

  3. Server Deployment with GPU:

  4. Offline Batch Processing:

In conclusion, deployment metrics analysis reveals that model selection should be guided by the specific constraints and requirements of the deployment environment. The custom BananaLeafCNN model demonstrates exceptional efficiency across numerous deployment metrics, making it an excellent choice for resource-constrained environments, while larger models remain appropriate for scenarios where computational resources are less limited.

8. Discussion

This section synthesizes insights from our comprehensive analysis of various CNN architectures for banana leaf disease diagnosis. We move beyond reporting results to discuss implications for model selection, robustness strategies, and real-world implementation challenges.

8.1 Architecture Performance Insights

8.1.1 Transfer Learning Efficacy

Our experiments with pre-trained models (ResNet50, VGG16, DenseNet121, MobileNetV3, EfficientNetB3) versus our custom BananaLeafCNN reveal nuanced trade-offs in transfer learning efficacy for agricultural applications:

  1. Feature Transferability Gap: While pre-trained models demonstrated strong baseline performance, we observed diminishing returns in their ability to capture banana disease-specific features. Particularly for conditions like Black Sigatoka, which presents subtle early-stage symptoms, pre-trained models often leveraged general texture patterns rather than disease-specific markers. This suggests a domain gap between general object recognition (ImageNet) and specialized agricultural disease diagnosis.

  2. Fine-tuning Efficiency Disparity: Fine-tuning efficiency varied dramatically across architectures, with EfficientNetB3 requiring 2.3× fewer epochs to converge compared to VGG16. This suggests that architectures with more sophisticated feature hierarchies retain greater adaptability for domain transfer, a critical consideration for agricultural applications where specialist-annotated training data may be limited.

  3. Custom Architecture Advantages: BananaLeafCNN, despite being significantly smaller (0.2M parameters), achieved competitive accuracy (92.7%) by incorporating domain-specific architectural choices. The focused design eliminated redundant feature extraction pathways irrelevant to leaf disease manifestation patterns, demonstrating that domain-informed architecture design can partially compensate for the advantages of extensive pre-training.

  4. Layer-wise Transfer Analysis: Our experiments with progressive fine-tuning showed that the most critical adaptation for pre-trained models occurs in the mid-level convolutional layers (layers 3-4 in ResNet50), where feature representations transition from generic to domain-specific. This finding suggests that hybrid transfer approaches—freezing early layers while extensively retraining middle and late layers—could optimize the pre-trained vs. custom model trade-off.

8.1.2 Model Complexity Trade-offs

Our analysis reveals several key insights regarding model complexity:

  1. Inverted Parameter-Performance Relationship: We observed that parameter count correlates poorly with disease classification performance beyond a critical threshold. The 0.2M-parameter BananaLeafCNN (92.7% accuracy) outperformed the 134M-parameter VGG16 (91.2% accuracy), representing a 670× reduction in parameters with a 1.5 percentage point accuracy improvement. This inverted relationship challenges the conventional wisdom that larger models necessarily perform better for specialized tasks.

  2. Efficiency Optimization Ceiling: Our ablation studies revealed that models under 1M parameters (BananaLeafCNN, pruned MobileNetV3) encountered performance instability, while models above approximately 5M parameters (ResNet50, EfficientNetB3) showed negligible gains despite massive parameter increases. This suggests an "efficiency optimization ceiling" specific to the banana disease classification domain—a sweet spot where model capacity aligns with task complexity.

  3. Architecture-Specific Efficiency Ratios: When evaluating models using our performance-to-size ratio metric (accuracy percentage points per 100K parameters), we found dramatic differences: BananaLeafCNN (46.35), MobileNetV3 (2.19), EfficientNetB3 (0.87), ResNet50 (0.39), DenseNet121 (1.31), and VGG16 (0.07). This 660× efficiency range between the best and worst models highlights the critical importance of architecture selection for resource-constrained agricultural applications.

  4. Inference Complexity Considerations: Beyond parameter count, we found that architectural choices significantly impact computational complexity during inference. EfficientNetB3, despite having fewer parameters than ResNet50, demonstrated higher CPU latency due to its compound scaling approach and more complex activation functions. For deployment scenarios, FLOPS and memory access patterns proved more predictive of real-world performance than raw parameter counts.

8.2 Robustness Implications

8.2.1 Field Condition Considerations

Our robustness analysis provides critical insights for field deployment:

  1. Environmental Perturbation Mapping: Our systematic evaluation of seven perturbation types revealed that real-world environmental factors map differently to model performance. Brightness and contrast variations (mimicking different times of day and weather conditions) caused average accuracy drops of 69.8% and 67.8% respectively, while geometric transformations (mimicking different viewing angles) caused a 30.2% drop. This mapping allows anticipation of performance variability under specific field conditions.

  2. Localized Adaptation Requirements: Models exhibited regional performance differences that correspond to real-world agricultural regions. DenseNet121 maintained higher accuracy under low-light conditions (similar to plantation understory environments), while MobileNetV3 performed better under high-brightness conditions (similar to direct sunlight scenarios). This suggests that model selection should consider the specific environmental conditions of the deployment region.

  3. Temporal Robustness Factors: Our analysis of time-of-day simulation (brightness variation combined with color temperature shifts) revealed that all models perform best during mid-day conditions, with accuracy degrading by an average of 15.2% during early morning or late afternoon simulated lighting. This temporal performance variation has direct implications for when farmers should capture images for most reliable diagnosis.

  4. Environmental Preprocessing Strategies: Based on our robustness findings, we identified critical preprocessing interventions to mitigate environmental variability. Specifically, contrast normalization improves average model performance by 24.6% under variable lighting, and targeted denoising improves performance by 18.3% under low-light conditions. These preprocessing strategies offer practical pathways to enhance environmental adaptability without model retraining.

8.2.2 Robustness-Accuracy Trade-offs

Our investigation revealed complex relationships between robustness and accuracy:

  1. Architectural Robustness Characteristics: Architecture design choices substantially impact robustness profiles independent of raw accuracy. MobileNetV3, with its depthwise separable convolutions, demonstrated superior resilience to noise perturbations despite lower baseline accuracy than DenseNet121. This suggests that certain architectural patterns inherently favor robustness across perturbation types.

  2. The Robustness-Accuracy Tension: We observed a general tension between optimization for accuracy versus robustness. Models with the highest baseline accuracy often demonstrated the steepest performance degradation under perturbations. For example, EfficientNetB3 achieved 94.1% baseline accuracy but experienced a 58.5% average relative accuracy drop under perturbations, while BananaLeafCNN achieved 92.7% baseline with a 68.1% average drop. This highlights the importance of evaluating models beyond ideal-condition performance.

  3. Regularization Effects on Robustness: Our ablation experiments revealed that regularization techniques impact robustness asymmetrically across perturbation types. Dropout (30%) improved noise robustness by 7.3% while decreasing occlusion robustness by 2.1%, whereas batch normalization improved geometric transformation robustness by 12.4% while minimally affecting other perturbation types. This suggests that targeted regularization strategies should be employed based on anticipated deployment conditions.

  4. Training Approaches for Improved Resilience: We identified that data augmentation strategies aligned with expected perturbations substantially improve robustness. Models trained with targeted augmentation (specific to the deployment environment's conditions) showed an average 28.7% reduction in accuracy degradation under corresponding perturbations. This demonstrates that training methodology, not just architecture selection, critically influences field robustness.

8.3 Practical Deployment Considerations

8.3.1 Resource-Constrained Applications

Our deployment metrics analysis reveals crucial considerations for field implementation:

  1. Model Selection Framework: Based on our comprehensive benchmarking, we developed a decision framework for model selection under resource constraints. For devices with under 1GB RAM, BananaLeafCNN provides the optimal balance of accuracy (92.7%) and peak memory usage (52MB). For devices with moderate computational capability but strict storage limitations, MobileNetV3 offers the best compromise between CPU latency (72ms) and model size (16MB).

  2. Export Format Optimization: Our cross-format comparison demonstrated that ONNX consistently provides 5-10% latency improvements over PyTorch native models across all architectures, with the improvement magnitude inversely proportional to model size. This optimization comes with minimal implementation complexity, making it a crucial "free" performance enhancement for resource-constrained deployments.

  3. Batch Processing Strategies: For scenarios requiring batch processing (e.g., extension officers collecting multiple images for later analysis), optimizing batch size dramatically improves throughput. BananaLeafCNN achieves optimal CPU throughput at batch size 4 (250 samples/s), while MobileNetV3 peaks at batch size 8 (246 samples/s). These optimization points provide 3.1× and 2.8× throughput improvements over single-sample processing, respectively.

  4. Hardware-Specific Optimization Opportunities: Our platform-specific analysis revealed that quantization to 16-bit precision provides a 1.8× speed improvement on CPU with only a 0.6 percentage point accuracy reduction across models. This represents a particularly valuable optimization for mobile and edge deployments where specialized hardware acceleration may be unavailable.

8.3.2 Real-world Implementation Challenges

Beyond technical metrics, several practical considerations emerge for field implementation:

  1. Integration with Agricultural Workflows: Our analysis highlights the need to align model deployment with existing agricultural practices. The 72-115ms inference latency (MobileNetV3 and BananaLeafCNN) enables real-time diagnosis during typical field scouting activities, whereas the 784ms latency of VGG16 would disrupt the typical inspection rhythm. This temporal integration with workflow patterns is as important as raw technical performance.

  2. User Interface Implications: Our robustness findings directly inform UI design requirements. The identification of critical failure thresholds (e.g., rotations beyond 5°, blur with kernel size > 3) suggests that camera guidance overlays should be incorporated to help users avoid these conditions. Additionally, confidence thresholds should trigger user warnings when environmental conditions approach model limitation boundaries.

  3. Farmer Accessibility Factors: The dramatic differences in model size (0.8MB for BananaLeafCNN vs. 512MB for VGG16) have direct implications for technology accessibility. In regions with limited mobile data connectivity, download size becomes a critical adoption barrier. Our analysis suggests that models exceeding 50MB would face significant deployment friction in rural agricultural regions with constrained connectivity.

  4. On-device vs. Cloud Deployment Trade-offs: The 34× GPU acceleration factor for BananaLeafCNN suggests that cloud deployment with GPU acceleration could process approximately 3,831 images per second compared to 112 images per second on-device. However, this theoretical advantage must be balanced against connectivity limitations, data costs, and the 2-3 second round-trip latency typical in rural agricultural settings, which would negate the raw inference speed advantage.

In conclusion, our discussion highlights that effective banana leaf disease diagnosis systems require careful consideration of transfer learning efficacy, model complexity trade-offs, environmental robustness factors, and practical deployment constraints. The optimal solution involves not simply selecting the most accurate model, but rather identifying the architecture and deployment strategy that best balances performance, robustness, and resource efficiency for the specific agricultural context.

10. Conclusion

This research has presented a systematic, multi-faceted evaluation of deep learning models for banana leaf disease classification, moving beyond standard accuracy metrics to consider robustness under variable field conditions and performance within practical deployment constraints. Through extensive comparative analysis, we have developed insights that bridge the gap between laboratory performance and real-world agricultural implementation.

10.1 Summary of Key Findings

Our comprehensive analysis of six CNN architectures—BananaLeafCNN (custom), ResNet50, VGG16, DenseNet121, MobileNetV3, and EfficientNetB3—revealed several significant findings:

  1. Classification Performance: All evaluated architectures achieved acceptable baseline accuracy (>90%) under controlled conditions, with EfficientNetB3 demonstrating the highest accuracy (94.1%) followed closely by our custom BananaLeafCNN (92.7%) despite the latter's dramatically simpler architecture.

  2. Robustness Profiles: Models exhibited distinctive vulnerability patterns across perturbation types, with brightness variations and blur causing the most severe degradation (average accuracy drops of 69.8% and 73.2% respectively). Architecture design choices substantially influenced robustness independently of baseline accuracy, as evidenced by MobileNetV3's superior resilience to noise perturbations despite its lower baseline accuracy compared to some competitors.

  3. Parameter Efficiency: Our custom BananaLeafCNN achieved remarkable efficiency with only 0.2M parameters—a 670× reduction compared to VGG16 (134M)—while maintaining competitive accuracy. This inverted parameter-performance relationship challenges the conventional wisdom that larger models necessarily perform better for specialized agricultural tasks.

  4. Deployment Metrics: BananaLeafCNN demonstrated exceptional deployment characteristics, including a 34× GPU acceleration factor, 115ms CPU inference latency, and 52MB peak memory usage. ONNX format consistently provided 5-10% latency improvements across architectures, offering a "free" performance enhancement for resource-constrained deployments.

  5. Environmental Adaptability: Models showed varied adaptability to environmental conditions, with DenseNet121 maintaining higher accuracy under low-light conditions while MobileNetV3 performed better under high-brightness scenarios. Preprocessing interventions including contrast normalization and targeted denoising offered critical improvements (24.6% and 18.3% respectively) under variable conditions.

  6. Batch Processing Optimization: Model-specific batch size optimization revealed significant throughput improvements, with BananaLeafCNN achieving optimal CPU throughput at batch size 4 (250 samples/s) and MobileNetV3 peaking at batch size 8 (246 samples/s)—representing 3.1× and 2.8× improvements over single-sample processing.

10.2 Theoretical and Practical Implications

Our findings have both theoretical and practical implications for agricultural computer vision:

10.2.1 Theoretical Implications

  1. Domain Specialization vs. Transfer Learning: Our results demonstrate that domain-specialized architectures can achieve comparable or superior performance to general-purpose networks with orders of magnitude fewer parameters, suggesting that the benefits of transfer learning may be overstated for specialized agricultural applications.

  2. The Robustness-Accuracy Tension: We identified a fundamental tension between optimization for accuracy versus robustness. Models with the highest baseline accuracy often demonstrated the steepest performance degradation under perturbations, highlighting the importance of robustness as a first-class evaluation metric alongside accuracy.

  3. Architecture-Specific Robustness Profiles: Our systematic perturbation analysis revealed that architecture design choices impart distinctive robustness characteristics independent of baseline accuracy. This suggests that robustness should be considered an intrinsic architectural property rather than simply a byproduct of general performance.

  4. Efficiency Optimization Ceiling: The study revealed an "efficiency optimization ceiling" specific to the banana disease classification domain—a parameter threshold beyond which additional model capacity yields diminishing or negative returns. This finding challenges the trend toward increasingly larger models in computer vision research.

10.2.2 Practical Implications

  1. Deployment-Oriented Model Selection: Our findings support a context-sensitive approach to model selection based on specific deployment requirements. For mobile applications, BananaLeafCNN or MobileNetV3 provide the optimal balance of accuracy, efficiency, and robustness, while server deployments with GPU availability may benefit from EfficientNetB3's higher accuracy.

  2. Environmental Preprocessing Strategies: The identification of critical preprocessing interventions provides practical pathways to enhance model performance in variable field conditions without requiring architectural changes or retraining.

  3. Export Format Optimization: Our cross-format comparison demonstrates that ONNX consistently provides latency improvements across all architectures, offering a practical optimization strategy for all deployment scenarios.

  4. Accessibility Considerations: The dramatic differences in model size (0.8MB for BananaLeafCNN vs. 512MB for VGG16) have direct implications for technology accessibility in regions with limited connectivity, suggesting that parameter efficiency should be a primary consideration for agricultural applications.

10.3 Research Contributions

This study makes several significant contributions to the field of agricultural computer vision:

  1. Multi-Faceted Evaluation Framework: We have established a comprehensive framework for evaluating deep learning models that considers not only ideal-case accuracy but also robustness, efficiency, and deployment characteristics—providing a template for more holistic model assessment in agricultural applications.

  2. BananaLeafCNN Architecture: Our custom-designed architecture demonstrates that domain-informed design choices can create highly efficient models for specialized agricultural tasks, offering an alternative to the transfer learning paradigm that dominates current approaches.

  3. Systematic Perturbation Analysis: By quantifying model resilience across seven perturbation types that simulate field conditions, we have provided a methodology for anticipating real-world performance degradation and identifying critical failure thresholds.

  4. Deployment-Oriented Benchmarking: Our detailed analysis of inference latency, memory usage, batch processing optimization, and export format performance establishes benchmarks for evaluating deployment feasibility across computational environments.

  5. Context-Specific Model Selection Framework: Rather than identifying a single "best" model, we have developed evidence-based guidelines for selecting appropriate architectures based on specific agricultural deployment scenarios and resource constraints.

10.4 Future Research Directions

While our research provides comprehensive insights into current CNN architectures for banana leaf disease classification, several promising directions for future work emerge:

  1. Semi-Supervised Learning: Investigating semi-supervised approaches to reduce reliance on large annotated datasets, which remain a constraint for specialized agricultural applications.

  2. Multi-Modal Fusion: Exploring the integration of multiple data modalities (RGB, multispectral, thermal) to enhance classification reliability under variable field conditions.

  3. Temporal Disease Progression: Developing models that can track disease progression over time, providing early warning capabilities before symptoms become visually apparent.

  4. Explainable AI Methods: Incorporating explainability techniques to help agricultural practitioners understand model decisions and build trust in automated diagnosis systems.

  5. On-Device Learning: Investigating federated and on-device learning approaches that can adapt to local conditions without requiring constant connectivity or centralized retraining.

In conclusion, our research demonstrates that effective banana leaf disease diagnosis systems require careful consideration of the interplay between architecture design, robustness characteristics, and deployment constraints. The optimal solution involves not simply selecting the most accurate model, but rather identifying the architecture and deployment strategy that best balances performance, robustness, and resource efficiency for the specific agricultural context. The multi-faceted evaluation framework and deployment recommendations presented in this study provide a foundation for implementing practical, accessible disease diagnosis systems that can function effectively under the variable conditions of real-world agricultural environments.